Systems and methods for search modification

ABSTRACT

In some embodiments, apparatuses and methods are provided herein useful to presenting search diagnostic tools in concert with a website. In some embodiments, a system for presenting search diagnostic tools in concert with a website comprises a server, wherein the server stores code for a website including a first code set including executable code for rendering the website and a second code set including executable code for rendering a user interface integrated with the rendering of the website including search diagnostic tools for the website, and a control circuit configured to receive a search query, receive results for the search query, output signals to cause rendering of the website, wherein the website includes the results for the search query, and output signals to cause rendering of the user interface including the search diagnostic tools such that the user interface is presenting with the search results for the search query.

This application is a continuation of U.S. application Ser. No.16/670,127, filed Oct. 31, 2019, which claims the benefit of U.S.Provisional Application No. 62/754,928, filed Nov. 2, 2018, U.S.Provisional Application No. 62/754,910, filed Nov. 2, 2018, U.S.Provisional Application No. 62/754,992, filed Nov. 2, 2018, U.S.Provisional Application No. 62/755,000, filed Nov. 2, 2018, U.S.Provisional Application No. 62/754,981, filed Nov. 2, 2018, which areincorporated by reference in their entirety herein.

TECHNICAL FIELD

This invention relates generally to websites presented via a web browserand, more particularly, search functionality of websites presented viaweb browsers.

BACKGROUND

Many websites include search functions for users. The search functionsallow users to search, based on search queries, for content on thewebsite and/or external to the website. For example, the search functionof a retailer's website allows users to search for products offered forsale by the retailer and the search function of a search provider (e.g.,Google or Yahoo) allows users to search for content across numerouswebsites. In either case (i.e., searching within a website and/or searchacross websites), the search function is typically capable of receivingcomplex search queries using varying operators in addition to plainlanguage. However, typical users are most comfortable using plainlanguage for search queries and, consequently, use plain language whensearching via the search function. Unfortunately, plain language doesnot always yield the most relevant search results. Additionally, even ifa user utilizes complex operators for his or her search, he or she ismay not find what he or she is looking for.

Systems exist that allow computer programmers to alter the functionalityof search engines to enhance the results received by users. However,such systems are complex and require specially trained engineers toalter the functionality of the search engine. Because existing systemsare so complex, few people have the skill necessary to execute suchchanges which makes such search enhancement difficult and limited to fewpeople. Consequently, a need exists for systems, methods, andapparatuses that make enhancing searches easier and accessible to morepeople.

BRIEF DESCRIPTION OF THE DRAWINGS

Disclosed herein are embodiments of systems, apparatuses, and methodspertaining to presenting search diagnostic tools in concert with awebsite. This description includes drawings, wherein:

FIG. 1 depicts a webpage being presented via web browser 100 in concertwith search diagnostic tools 106, according to some embodiments;

FIG. 2 is a block diagram of a system 200 for presenting searchdiagnostic tools in concert with a website, according to someembodiments;

FIG. 3 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments;

FIG. 4 depicts a webpage being presented via a web browser 400 beforeprioritization functionality has been utilized, according to someembodiments;

FIG. 5 depicts a webpage being presented via a web browser 500 afterprioritization functionality has been utilized, according to someembodiments;

FIG. 6 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments;

FIG. 7 depicts a webpage being presented via a web browser 700 before afilter tool 714 has been utilized, according to some embodiments;

FIG. 8 depicts a webpage being presented via a web browser 800 after afilter tool 814 has been utilized, according to some embodiments;

FIG. 9 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments:

FIG. 10 depicts a webpage being presented via a web browser 1000presenting an explain dialogue 1006, according to some embodiments;

FIG. 11 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments;

FIG. 12 depicts a webpage being presented via a web browser 1200including a query tool 1204, according to some embodiments; and

FIG. 13 is a flow chart depicting example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments.

Elements in the figures are illustrated for simplicity and clarity andhave not necessarily been drawn to scale. For example, the dimensionsand/or relative positioning of some of the elements in the figures maybe exaggerated relative to other elements to help to improveunderstanding of various embodiments of the present invention. Also,common but well-understood elements that are useful or necessary in acommercially feasible embodiment are often not depicted in order tofacilitate a less obstructed view of these various embodiments of thepresent invention. Certain actions and/or steps may be described ordepicted in a particular order of occurrence while those skilled in theart will understand that such specificity with respect to sequence isnot actually required. The terms and expressions used herein have theordinary technical meaning as is accorded to such terms and expressionsby persons skilled in the technical field as set forth above exceptwhere different specific meanings have otherwise been set forth herein.

DETAILED DESCRIPTION

Generally speaking, pursuant to various embodiments, systems,apparatuses, and methods are provided herein useful to presenting searchdiagnostic tools in concert with a website. In some embodiments, asystem for presenting search diagnostic tools in concert with a websitecomprises a server, wherein the server stores code for a websiteincluding a search function for users of the website, wherein the codefor the web site includes a first code set and a second code set,wherein the first code set includes executable code for rendering thewebsite including the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchdiagnostic tools for the website, and a control circuit, wherein thecontrol circuit is configured to receive, from a user, a search query,receive, from a search engine, results for the search query, outputbased on execution of the first code set, signals to cause rendering ofthe website, wherein the website includes the results for the searchquery, and output, based on execution of the second code set, signals tocause rendering of the user interface including the search diagnostictools such that the user interface including the search diagnostic toolsis presenting with the search results for the search query.

INTRODUCTION

As previously discussed, many websites include search functions thatallow users of the website to search for content. While the examplesprovided herein discuss search features that allow users to search forcontent contained within the website, it should be noted that thesystems, methods, and apparatuses described herein can be used tofacilitate and improve searches designed to find content across multiplewebsites. Typically, users utilize the search function using plainlanguage. For example, a user of a retailer's website may enter “shoes”as a query when he or she is searching for shoes. The search results forthis query may include shoes, as well as other products, such as shoepolish, shoe laces, boots, etc. If the user is interested in viewingonly shoes, these other products may clutter the search and prevent theuser from finding what he or she is looking for. Currently, if theretailer wanted to alter the results presented (i.e., the search resultsfor the search query), the retailer would need to modify thefunctionality of the search engine. For example, the retailer couldmodify the ranking data for products in the search engine, the searchalgorithm, the data for products, configuration of pages, etc. Whilethis may improve the search results, such modification is difficult toperform and requires a specially trained software engineer.

Embodiments of the systems, methods, and apparatuses described hereinseek to minimize, or eliminate, the need for a specially trainedsoftware engineer to alter the search results. Instead, in someembodiments, a user interface is presented in concert with the website.The user interface includes search diagnostic tools for the website. Insome embodiments, a user with only the ability to navigate a websitewould be able to manipulate searches using the search diagnostic tools.

In one example embodiment, the search diagnostic tools include a numberof selections or buttons presented with search results for a searchquery. By selecting the selections or buttons, as well as performingother selections in some cases, a user can cause search queries to bemodified. As one example, described in more detail below, the user canselect a search result and have that search result “forced” to the topof the search results. In this way, when another user enters a searchquery, the search result “forced” to the top will be presented at thetop of the search results. The user can “force” this search result tothe top without modification to the search engine through the use of thesearch diagnostic tools presented in concert with the website.

The discussion of FIGS. 1-3 provides an overview of systems, methods,and apparatuses for presenting search diagnostic tools in concert with awebsite, the discussion of FIGS. 4-10 describes example ways that searchresults can be manipulated via the search diagnostic tools, thediscussion of FIGS. 10-11 describes information that can be generated bythe search diagnostic tools, and the discussion of FIGS. 12-13 describesan example automation that can be achieved via the search diagnostictools.

Overview—Search Diagnostic Tools

FIG. 1 depicts a webpage being presented via web browser 100 in concertwith search diagnostic tools 106, according to some embodiments. In theexample depicted in FIG. 1, the website is for a retailer (e.g., a partyselling goods and/or services online). While the example websiteprovided in FIG. 1 is for a retailer, the systems, methods, andapparatuses described herein can be used with any type of website thatincludes a search function.

The website depicts a number of search results (i.e., products 104 inthis example). The search results were generated based on a search queryinput by a user. For example, if the search query is “milk,” the searchresults are products 104 fitting the description “milk” or associatedwith the search query “milk.” In addition to the search results, thewebsite includes a user interface integrated with the website (i.e., theuser interface is presented in concert with the website). The userinterface includes search diagnostic tools 106 (including a search querytool 108). The search diagnostic tools 106 allow a user to manipulatesearches in an effort to alter the results of a search query.Specifically, in many instances, a user can modify a search query viathe search diagnostic tools 106. As depicted in FIG. 1, at least some ofthe search diagnostic tools 106 are associated with ones of the searchresults. That is, when appropriate based on the functionality of aspecific search diagnostic tool, each of the search results is presentedwith associated search diagnostic tools 106.

In some embodiments, the search diagnostic tools 106 are only presentedto specific users. For example, a general user of the website (e.g., acustomer) would see the search results but not the search diagnostictools. In such embodiments, only authorized users would be presentedwith the search diagnostic tools 106. Such access control can beachieved by use of any suitable credentials (e.g., user names andpasswords, use of virtual private networks (VPNs), network accesscontrols, etc.). In one embodiment, a user is prompted for credentialsupon receipt of a specific entry from the user (e.g. a predeterminedpattern of keystrokes or “hotkeys”). Upon receipt of the specific entryfrom the user, the website presents an authorization prompt which, forexample, allows the user to input his or her credentials.

As one example use case, an authorized user (e.g., an employee of theretailer), can review search results for search queries to identifysearch results that he or she feels do not fit well with the searchqueries. In such a use case, the employee can enter a search query(e.g., “bikes”) and the website presents search results (i.e., products104) that are returned based on the search query “bikes.” The user canthen select that the user interface be presented (e.g., by entering hisor her credentials). The web browser 100 presents the user interface,including the search diagnostic tools 106, in concert with the website.The user can manipulate the search results via the search diagnostictools. As one example, if one of the search results is a tricycle andthe user feels that the tricycle would not be a useful search result forthe query “bikes,” the user can remove the tricycle from the searchresults. Once this action has been performed, when other users (e.g.,customers) enter the search query “bikes,” the tricycle will no longerbe included in the search results. That is, the user has successfullymanipulated the search functionality of the website, via the searchdiagnostic tools 106, to exclude the tricycle when other users (e.g.,customers) search for bikes on the website. The user can manipulate thesearch without the need for any special training or knowledge. Ideally,any user that is able to use a web browser can utilize the searchdiagnostic tools to alter the results of a search. This is possiblebecause the search engine itself need not be modified to accomplish thedesired results.

While the discussion of FIG. 1 provides an overview of presenting searchdiagnostic tools in concert with a website, the discussion of FIG. 2describes a system for presenting search diagnostic tools in concertwith a website.

FIG. 2 is a block diagram of a system 200 for presenting searchdiagnostic tools in concert with a website, according to someembodiments. The system 200 includes a control circuit 202, a database204, user device(s) 206, a key value store 210, a server 212, and asearch engine 218. The control circuit 202, the database 204, the userdevice(s) 206, the key value store 210, the server 212, and the searchengine 218 are communicatively coupled to at least some of the othercomponents via a network 208. The network 208 can include multiplenetworks, such as intranets and internets (e.g., the Internet). That is,the network 208 is presented to indicate that the components depicted inFIG. 2 can communicate with one another, even though the network 208 maynot be a single network, as depicted in FIG. 2. Additionally, althoughthe control circuit 202, the database 204, the user device(s) 206, thekey value store 210, the server 212, and the search engine 218 aredepicted in FIG. 2, all of these components may not be required for allembodiments. For example, in some embodiments, the functionality of thedatabase 204 and the key value store 210 may be merged into a singlecomponent, the key value store 210 may not be necessary, one or more ofthe components may be integrated into a single component, etc.

The server 212 stores code for a website. The server 212 can be atraditional web server that includes memory, processing capabilities,etc. and conforms to the HTTP protocol. In some embodiments, the server212 can store multiple code sets. In the example depicted in FIG. 2, theserver 212 stores two code sets: a first code set 214 and a second codeset 216. Each of the two code sets includes executable code. The firstcode set 214 includes executable code for rendering the websiteincluding a search function for use with the website (e.g., to searchfor content within the website and/or contained within other websites).The second code set 216 includes executable code for rendering a userinterface integrated with the rendering of the website. The userinterface includes search diagnostic tool, as will be discussed in moredetail below. Although FIG. 2 depicts the server 212 as storing only twocode sets, embodiments are not so limited. As one example, the serve 212could contain one code set including executable code for rendering thewebsite and multiple other code sets for rendering different searchdiagnostic tools (e.g., each other code set could include executablecode for rendering and/or allowing use of one or more search diagnostictools). In such embodiments, the presentation of the search diagnostictools can be controlled so that access to the search diagnostic tools isnot binary (i.e., a user either does, or does not, have access to thesearch diagnostic tools). For example, different types of users may haveaccess to different ones of the search diagnostic tools. A web browserrenders the executable code based on the server 212 outputting code,data and/or instructions (e.g., j son, XML, or other data interchangemethods) instructing the web browser to present the user interface andthe content.

The user device(s) 206 can be any device(s) suitable for presentingwebsites (e.g., desktop computers, laptop computers, tablets, smartphones, automotive infotainment systems, etc.). The user device(s) 206can be used by general users (e.g., visitors of the website) to entersearch queries and review search results. The user device(s) 206 canalso be used by authorized users (i.e., users having permission orcredentials required to view the user interface including the searchdiagnostic tools), such as employees associated with the website,contractors, or other third parties.

The control circuit 202 can comprise a fixed-purpose hard-wired hardwareplatform (including but not limited to an application-specificintegrated circuit (ASIC) (which is an integrated circuit that iscustomized by design for a particular use, rather than intended forgeneral-purpose use), a field-programmable gate array (FPGA), and thelike) or can comprise a partially or wholly-programmable hardwareplatform (including but not limited to microcontrollers,microprocessors, and the like). These architectural options for suchstructures are well known and understood in the art and require nofurther description here. The control circuit 202 is configured (forexample, by using corresponding programming as will be well understoodby those skilled in the art) to carry out one or more of the steps,actions, and/or functions described herein.

By one optional approach the control circuit 202 operably couples to amemory. The memory may be integral to the control circuit 202 or can bephysically discrete (in whole or in part) from the control circuit 202as desired. This memory can also be local with respect to the controlcircuit 202 (where, for example, both share a common circuit board,chassis, power supply, and/or housing) or can be partially or whollyremote with respect to the control circuit 202 (where, for example, thememory is physically located in another facility, metropolitan area, oreven country as compared to the control circuit 202).

This memory can serve, for example, to non-transitorily store thecomputer instructions that, when executed by the control circuit 202,cause the control circuit 202 to behave as described herein. As usedherein, this reference to “non-transitorily” will be understood to referto a non-ephemeral state for the stored contents (and hence excludeswhen the stored contents merely constitute signals or waves) rather thanvolatility of the storage media itself and hence includes bothnon-volatile memory (such as read-only memory (ROM) as well as volatilememory (such as an erasable programmable read-only memory (EPROM)) orrandom access memory (RAM). The control circuit 202 can include anhardware and/or software necessary to conduct the operations describedherein.

Without implying any positional limitations, the control circuit 202 canbe thought of as residing “between” the user device(s) 206 and thesearch engine 218. That is, although the control circuit 202 need not bephysically located between the user device(s) 206 and the search engine218, communications from the user device(s) 206 to the search engine 218pass through the control circuit 202. For example, when a user submits asearch query via a user device 206, the control circuit 202 receives thesearch query and passes the search query to the search engine 218.

When a user device 206 accesses the website, the server 212 serves thewebsite to the user device 206. In some embodiments, the presentation ofthe website is dependent upon the user accessing the website. In oneembodiment, a general user will only see the website (i.e., execution ofthe first code set 214). Further, only authorized users will see thewebsite and the user interface (i.e., execution of the first code set214 and the second code set 216). In this embodiment, the controlcircuit 202 will only output signals to cause rendering of the websitebased on execution of the first code set 214 for a general user and thecontrol circuit 202 will output signals (e.g., via a markup languagesuch as XML or j son and/or code such JavaScript) to cause rendering ofthe website based on the first code set 214 and the user interface basedon the second code set 216. In the aforementioned embodiment, the user'suser device 206 does not control access to the first code set 214 and/orthe second code set 216. In some instances, this embodiment may bepreferential because the amount of data transferred to, and handled by,the user's user device 206 is limited and it may afford greater accesscontrol. In other embodiments, all user device(s) 206 will receive thesignals to cause rendering of the first code set 214 and the second codeset 216. However, the user's user device 206 will control whether theuser interface is rendered based on whether the user is an authorizeduser. In some instances, these embodiments may be preferential as theymay reduce computational overhead for the control circuit 202. In eitherof the above-noted embodiments, access to the user interface can becontrolled using access controls. For example, a user device 206 may notpresent, or receive signals to cause rendering of, the user interfaceunless authorization information is received. For example, theauthorization information can include a username and password, a networkthrough which the user accesses the server, biometric information, orany other suitable information.

In some embodiments, utilization of one or more of the search diagnostictools may cause modification of a search query (e.g., adding and/ordeleting content from the search query). As previously noted, anddiscussed in more detail below, one of the search diagnostic tools mayallow a user to “force” a search result to the top of the searchresults. In this example, utilization of such a search diagnostic toolwould cause a query to be modified before it is passed to the searchengine 218. For example, the search query “tire” results in Product₁being returned by the search engine 218 and Product₁ being presentednear the bottom of the search results. If an authorized user feels thatProduct₁ should be presented higher in the search results, he or she can“force” Product₁ further up in the search results, for example to thefirst position, the first row, an upper section, a predominate position,etc. This change to the search result (i.e., the forcing of Product₁higher in the search results) is referred to as a modification to thesearch query. It is a modification to the search query because insteadof altering the search engine 218 to cause this change to the searchresult, the search query is modified (e.g., changes, enhanced, amended,etc.) to cause Product₁ to be returned at a higher position by thesearch engine 218. For example, in layman's terms, the search query“tire” may be modified to read “tire+[place Product₁ in a higherposition in the search results].” If the user saves this modification tothe search query, when another user (whether general or authorized)enters the search query, the modified search query is passed to thesearch engine 218.

In some embodiments, search queries and modified search queries arestored in the key value store 210. At a high level, the key value store210 can be thought of as a mapping between search queries and modifiedsearch queries. Continuing the example above, if the search query “tire”has been modified to read “tire+[place Product₁ in a higher position inthe search results],” the key value store 210 can include a mappingbetween the search query “tire” and the modified search query“tire+[place Product₁ in a higher position in the search results].”Accordingly, when a user searches for the search query “tire,” thecontrol circuit 202 accesses the key value store 210, retrieves themodified search query associated with the search query “tire,” andpasses the modified search query (i.e., “tire+[place Product₁ in ahigher position in the search results]”) to the search engine 218. Thesearch results returned by the search engine 218 are based on themodified search query.

In addition to the key value store 210, in some embodiments, the system200 also includes the database 204. While the key value store 210 storesassociations between the search queries and the modified search queries,the database 204 can store a more robust set of information regardingsearch queries and modified search queries (i.e., information related tothe search queries and the modified search queries). For example, thedatabase 204 can include previous modifications to search queries,indications of who made modifications to search queries, indications ofwhen modifications to search queries were made, information regardingsearch results associated with search queries and modified searchqueries, analytics information regarding the search queries and/ormodified search queries, etc. Additionally, in some embodiments, thedatabase 204 can be multiple databases. Additionally, each of themultiple databases can be tasked with storing different types ofinformation (e.g., a first database could store the previousmodifications to the search queries, a second database could storeindications of who made modifications to the search queries, etc.).

While the discussion of FIG. 2 describes a system for presenting searchdiagnostic tools in concert with a website, the discussion of FIG. 3describes example operation for presenting search diagnostic tools inconcert with a website.

FIG. 3 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments. The flow begins at block 302.

At block 302, code for a website is stored. For example, a server canstore the code for the website. The code for the website includesmultiple code sets. In one embodiment, the code for the website includestwo code sets: a first code set and a second code set. The first codeset includes executable code for rendering the website including asearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website. The user interface includes search diagnostic tools. The flowcontinues at block 304.

At block 304, a search query is received. For example, a control circuitcan receive the search query from a user. The search query is input bythe user and seeks to find content on the website and/or on otherwebsites related to the search query. The search query is passed to thesearch engine to generate search results. The flow continues at block306.

At block 306, search results are received. For example, the searchresults are received by the control circuit from the search engine. Thesearch results are based on the search query. The flow continues atblock 308.

At block 308, signals are output based on the first code set. Forexample, the control circuit can output the signals based on the firstcode set. The signals based on the first code set cause rendering of thewebsite by the user device. The rendering of the website includes thesearch results (i.e., the results of the search query). The flowcontinues at block 310.

At block 310, signals are output based on the second code set. Forexample, the control circuit can output the signals based on the secondcode set. The signals based on the second code set cause rendering ofthe user interface including the search diagnostic tools such that theuser interface including the search diagnostic tools is presented withthe results for the search query. For example, the search diagnostictools can be presented next to, on top of, superimposed on, near,adjacent to, and/or in conjunction with the search results.

While the discussion of FIGS. 1-3 provides background informationregarding systems, methods, and apparatuses for presenting searchdiagnostic tools in concert with a website, the discussion of FIGS. 4-13provides examples of the functionality of some of the search diagnostictools.

Example Search Diagnostic Tools Prioritization Tools (Boost and Bury):

As previously discussed, the discussion of FIGS. 1-3 provides backgroundinformation regarding systems, methods, and apparatuses for presentingsearch diagnostic tools in concert with a website. The discussion ofFIGS. 4-6 describes a prioritization tool of the search diagnostictools, according to some embodiments. FIG. 4 depicts a websitepresenting results for a search query before the prioritization tool isutilized and FIG. 5 depicts the website presenting results for thesearch query after the prioritization tool is utilized.

FIG. 4 depicts a webpage being presented via a web browser 400 beforeprioritization functionality has been utilized, according to someembodiments. The webpage is presenting search results based on a searchquery. In the example depicted in FIG. 4, the webpage is associated witha retailer's website and the search results are products 402 related tothe search query. The presentation of the website includes searchdiagnostic tools 404. The search diagnostic tools 404 are presented inconcert with the products 402.

The search diagnostic tools 404 important to this discussion are theprioritization tools. In the example depicted in FIG. 4, the searchdiagnostic tools 404 include two prioritization tools: a boost tool 412and a bury tool 408. Although the discussed boost tool 412 and bury tool408 are associated with the sixth product (i.e., Product 6) 410 and thesecond product (i.e., Product 2), respectively, it should be noted thateach of the products 402 includes prioritization tools.

The prioritization tools allow a user, such as an authorized user, toalter the order in which the products 402 are presented in the searchresults. As depicted in FIG. 4, each of the products 402 is presented inorder (i.e., Product 1 before Product 2, Product 2 before Product 3,Product 3 before Product 4, etc.). The order in which the products areordered in FIG. 4 is a consequence of a search engine ranking based onthe search query. Accordingly, the order in which the products 402 arepresented in FIG. 4 is representative of the search result orderingbefore the user has performed a prioritization action.

When the user utilizes the prioritization tools, he or she indicates amodification to the search query. As depicted in FIG. 4, and indicatedby the hashing, the user has selected the bury tool 408 for the secondproduct (i.e., Product 2) 406 and the boost tool 412 for the sixthproduct (i.e., Product 6) 410. Selection of the bury tool 408 will causethe second product (i.e., Product 2) to be presented lower in the searchresults (e.g., at the bottom of the search results, at the bottom of apage of search results, etc.). Selection of the boost tool 412 willcause the sixth product (i.e., Product 6) 412 to be presented higher inthe search results (e.g., at the top of the search results, in a higherpage of search results, etc.). The consequence of these selections isdepicted in FIG. 5.

Additionally, if the user selects the boost and/or bury tool again, itwill “turn off” the boost and/or bury. For example, if the user selectsthe boost tool 412 for the sixth product (i.e., Product 6) 410, thesixth product (i.e., Product 6) 410 will no longer be boosted.

FIG. 5 depicts a webpage being presented via a web browser 500 afterprioritization functionality has been utilized, according to someembodiments. That is, the web site presents the search results (i.e.,products 502 related to the search query) in an order that differs fromFIG. 4 because the user has utilized the prioritization tools.

As described above, the user has selected the bury tool for the secondproduct (i.e., Product 2) 506 and the boost tool 412 for the sixthproduct (i.e., Product 6) 510. That is, the user has indicated amodification to the search query that will cause the second product(i.e., Product 2) 506 to be presented lower in the search results andthe sixth product (i.e., Product 6) 510 to be presented higher in thesearch results. As depicted in FIG. 5, the sixth product (i.e., Product6) 510 now occupies a higher position (i.e., in the top left of thesearch results) in the search results and the second product (i.e.,Product 2) 506 now occupies a lower position in the search results(i.e., the bottom right position in the search results). Although FIG. 5depicts the boosted result (i.e., the sixth product) being moved to afirst position and the buried search result (i.e., the second product)moved to a last position, such is not required. For example, boosting asearch result may move the search result to a higher position, but notnecessarily the highest position and burying a search result may movethe search result to a lower position but not necessarily the lowestposition.

Use of the prioritization tools allows a user to alter the positioningof search results by modifying the search query. In some embodiments,because the search query is modified to alter the positioning of thesearch results, the search engine producing the search results does notneed to be altered. Consequently, the average user (i.e., someone withknowledge of only how to navigate the Internet) may be able toaccomplish such an alteration.

The search query can be modified in any suitable manner to achieve thisresult. In one example, the search query can be modified to includeindications of the search results to be prioritized. Continuing theexample described with respect to FIGS. 4 and 5, the search query can bemodified to include indications of the second product (i.e., Product 2)506 and the sixth product (i.e., Product 6) 510. The indication of theproduct can be a UPC, a SKU, or any other type of product identifier.Continuing the example, if the original unmodified search query is“[search query],” the modified search query may be “[searchquery]+[boost [UPC for the sixth product (i.e., Product 6) 510]]+[bury[UPC for the second product (i.e., Product 2) 506]].” This modifiedsearch query is passed to the search engine, resulting in the searchresults being presented via the website in an altered order. Put simply,after the user performs the prioritization modification, subsequentusers of the website that enter the search query will see the searchresults in an order as altered by the user.

In addition to using the prioritization tools with regard to specificsearch results, in some embodiments, users can utilize theprioritization tools to alter the positioning of search results in bulk.In such embodiments, users may be able to prioritize search resultsbased on any suitable grouping, such as by type, price, category, etc.

As previously discussed with respect to FIG. 2, a system for presentingsearch diagnostic tools in concert with a website can include a keyvalue store. In embodiments in which such a system includes a key valuestore, when the user utilizes the prioritization tool to modify thesearch query, a mapping is created in the key value store thatassociates the search query with the modified search query.

While the discussion of FIGS. 4 and 5 describes a website presentationbefore and after the prioritization tools have been used, the discussionof FIG. 6 describes example operations performed by a system forpresenting search diagnostic tools in concert with a website when theprioritization tools are used.

FIG. 6 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments. The flow begins at block 602.

At block 602, code for a website is stored. For example, a server canstore the code for the website. The code for the website includesmultiple code sets. In one embodiment, the code for the website includestwo code sets: a first code set and a second code set. The first codeset includes executable code for rendering the website including asearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website. The user interface includes search diagnostic tools. The flowcontinues at block 604.

At block 604, a mapping is stored. For example, a key value store canstore the mapping. The mappings are associations between search queriesand modified search queries. For example, a user (e.g., a personassociated with a website offering a search function) may feel that thesearch results returned for a search query are different than what he orshe would like. As will be discussed in detail below with respect toprioritization tools, the user can alter the way the search results arepresented via the website. When the user does so, he or she indicates amodification to the search query. The result is a modified search query.The modified search query is based on the original (i.e., unmodified)search query. The key value store stores a mapping between the original(i.e., unmodified) and modified search query. A single value may be acomposition of many modifications to the query. The flow continues atblock 606.

At block 606, a search query is received. For example, a control circuitcan receive the search query from a user. The search query is input bythe user and seeks to find content on the website and/or on otherwebsites related to the search query. The search query is passed to thesearch engine to generate search results. The flow continues at block608.

At block 608, search results are received. For example, the searchresults are received by the control circuit from the search engine. Thesearch results are based on the search query. The flow continues atblock 610.

At block 610, signals are output based on the first code set. Forexample, the control circuit can output the signals based on the firstcode set. The signals based on the first code set cause rendering of thewebsite by the user device. The rendering of the website includes thesearch results (i.e., the results of the search query). The flowcontinues at block 612.

At block 612, signals are output based on the second code set. Forexample, the control circuit can output the signals based on the secondcode set. The signals based on the second code set cause rendering ofthe user interface including the search diagnostic tools such that theuser interface including the search diagnostic tools is presented withthe results for the search query. For example, the search diagnostictools can be presented next to, on top of, superimposed on, near,adjacent to, and/or in conjunction with the search results. The searchdiagnostic tools include prioritization tools configured to controlprioritization of the results for the search query. The flow continuesat block 614.

At block 614, an indication of a modification to the search query isreceived. For example, the control circuit can receive the indication ofthe modification to the search query. The indication of the modificationto the search query is based on utilization of the prioritization tools.For example, the modification to the search query can be the boosting ofa search result or the burying of a search result. The flow continues atblock 616.

At block 616, a modified search query is generated. For example, thecontrol circuit can generate the modified search query. The modifiedsearch query is based on the search query and the indication of themodification of the search query. The search query can be modified inany manner suitable to obtain the desired prioritization. For example,an indication of the search result, and indication of a group of searchresults, an indication of a category of search results, an indication ofa priority for the search result, etc. can be incorporated into thesearch query to generate the modified search query. The flow continuesat block 618.

At block 618, the modified search query is transmitted for storage. Forexample, the control circuit can transmit the modified search query to akey value store for storage. Additionally, in some embodiments, thecontrol circuit can transmit the modified search query to a database(e.g., a SQL or NoSQL database) for storage. In such embodiments,additional information regarding the search query and the modifiedsearch query can be stored in the database. The flow continues at block620.

At block 620, submission of the search query is received from an enduser. For example, the control circuit can receive the submission of thesearch query from a user of the website, such as a general user or anauthorized user for testing and/or evaluation. The flow continues atblock 622.

At block 622, the modified search query is transmitted to the searchengine. For example, the control circuit can transmit the modifiedsearch query to the search engine. This step assumes that a modificationhas been performed to the search query submitted. For example, if amodified search query was previously generated for the specific searchquery that was submitted, instead of submitting the search query to thesearch engine, the modified search query is submitted. In someembodiments, the control circuit accesses the key value store toretrieve the modified search query associated with the search query. Insuch embodiments, the control circuit submits the modified search queryretrieved based on this access.

While the discussion of FIGS. 4-6 describes a prioritization tool of thesearch diagnostic tools, the discussion of FIGS. 7-9 describes a filtertool of the search diagnostic tools.

Filter Tools (Auto Clean):

The discussion of FIGS. 7-9 describes a filter tool of the searchdiagnostic tools. Generally, the filter tool allows users to quickly andeasily alter the results of a search based on scores associated witheach of the search results and, in some embodiments, the search query.FIG. 7 depicts a website presenting results for a search query beforethe filter tool is utilized and FIG. 8 depicts the website presentingresults for the search query after the filter tool has been utilized.

FIG. 7 depicts a webpage being presented via a web browser 700 before afilter tool 714 has been utilized, according to some embodiments. Thefilter tool 714 is a quick way for a user to “clean” (i.e., removelow-scoring search results) without having to manually prioritize (e.g.,“deprioritize”) each low-scoring search result. As depicted in FIG. 7,the filter tool 714 is set to an off position 716. Because the filtertool 714 is set to the off position 716, the website is presenting thesearch results received from the search engine without any filtering.Additionally, in some embodiments, the filter tool 714 allows the userto set a filter value via a set value selection 724. The filter value isthe value below which products will be excluded from the search results,as discussed in more detail below. In some embodiments, the filter valuecan be preset for example, to no value (no filtering), zero, five, orany other desired value.

The example website used for FIG. 7 is a retail website and the searchresults are products. As depicted in FIG. 7, the search engine hasreturned six search results based on a search query: a first product(i.e., Product 1) 702, a second product (i.e., Product 2) 704, a thirdproduct (i.e., Product 3) 706, a fourth product (i.e., Product 4) 708, afifth product (i.e., Product 5) 710, and a sixth product (i.e., Product6) 712. Each of the products has a score indicating the product'srelevance to the search query, such as an NDCG score, a Precision at Kscore, an MRR score, etc. As depicted in FIG. 7, the first product(i.e., Product 1) 702 has a score of ten, the second product (i.e.,Product 2) 704 has a score of three, the third product (i.e., Product 3)706 has a score of six, the fourth product (i.e., Product 4) 708 has ascore of seven, the fifth product (i.e., Product 5) 710 has a score ofone, and the sixth product (i.e., Product 6) 712 has a score of eight.

Instead of using the prioritization tools (as described with respect toFIGS. 4-6) to increase the priority of high-scoring product and/ordecrease the priority of low-scoring products, the user utilizes thefilter tool 714 to alter the results for the search query. In someembodiments, the user can indicate a filter value to be used by thefilter tool 714. For example, the user can choose to have all resultshaving a score below a specified value removed from the search results.In the example depicted between FIGS. 7 and 8, the user has selected afiltering value of six (i.e., all search results having a score belowsix will be removed from the search results). That is, the secondproduct (i.e., Product 2) 704 and the fifth product (i.e., Product 5)710 will be removed from the search results because both the secondproduct (i.e., Product 2) 704 and the fifth product (i.e., Product 5)710 have scores below six (i.e., scores of three and one, respectively).FIG. 8 depicts the consequence of utilizing the filter tool 714 with afilter value of six.

In some embodiments, the search diagnostic tools include a scoring tool722. The scoring tool 722 allows the user to set the score for theproduct. For example, as depicted in FIG. 7, the scoring tool 722 isassociated with the second product (i.e., Product 2) 704. Selection ofthe scoring tool 722 allows the user to set, alter, change, or otherwiseadjust the score for the product for which the scoring tool 722 isassociated. Although FIG. 7 depicts the scoring tool 722 as associatedonly with the second product (i.e., Product 2) 704, in some embodiments,each search result can have a scoring tool, similar to the scoring tool722. For example, selection of the score for a product may causepresentation of a scoring tool associated with the selected score forthe product. In some embodiments, the score for a product is associatedwith the search query. That is, if the search query is “Query₁” and theuser sets the score for the second product (i.e., Product 2) 704 to ten,the second product (i.e., Product 2) 704 will only have a score of tenfor the search query “Query₁.” If a subsequent user searches with thesearch query “Query₂,” the second product (i.e., Product 2) 704 may nothave a score of ten for the search query “Query₂.”

FIG. 8 depicts a webpage being presented via a web browser 800 after afilter tool 814 has been utilized, according to some embodiments. Asdepicted in FIG. 8, the filter tool 814 is set to an on position 816 andthe filter value has been set to six via a set value selection 822. Theuser can change the filter value, and thus the filtering of the searchresults, via the set value selection 822. When set to the on position816, the filter tool 814 removes all search results below the specifiedvalue. As previously discussed with respect to FIG. 7, the secondproduct (i.e., Product 2) 704 and the fifth product (i.e., Product 5)710 had scores below the specified value (i.e., six). Consequently,neither the second product (i.e., Product 2) 704 nor the fifth product(i.e., Product 5) 710 are included in the search results depicted inFIG. 8. Instead, these search results have been replaced with newproducts (i.e., a seventh product (i.e., Product 7) 818 and an eightproduct (i.e., Product 8) 820) both having scores above six. The seventhproduct (i.e., Product 7) 818 and the eighth product (i.e., Product 8)820 have scores of eight and nine, respectively. In one embodiment, thesearch results are “replaced with new products” by removing the oldproducts (i.e., the second product (i.e., Product 2) 704 and the fifthproduct (i.e., Product 5) 710), which causes other products returned forthe search query to move further up in the list. For example, asdepicted in FIG. 8, the second product (i.e., Product 2) 704 and thefifth product (i.e., Product 5) 710 no longer appear in the searchresults and two new products (i.e., the seventh product (i.e., Product7) 818 and the eighth product (i.e., Product 8) 820) have been moved upin the search results and now appear on the first page of the searchresults.

Although the example provided in FIG. 8 depicts new search resultsreplacing the removed search results, such is not required. That is,when the second product (i.e., Product 2) 704 and the fifth product(i.e., Product 5) 710 were removed, they may not be replaced with anyother products, leaving only the first product (i.e., Product 1) 702,the third product (i.e., Product 3) 706, the fourth product (i.e.,Product 4) 708, and the sixth product (i.e., Product 6) 712 from FIG. 7.Additionally, in some embodiments, the new products (i.e., the seventhproduct (i.e., Product 7) 818 and the eighth product (i.e., Product 8)820) may not be “new” in the sense that these two new products weren'tincluded in the search results previously. Rather, these new productsmay have been included in the original search results, but ordered insuch a manner that the two new products did not appear on the website aspresented in FIG. 7 (e.g., the user may have had to scroll down furtherto see these two new products).

Additionally, though not depicted in FIG. 8, in some embodiments, thefilter tool 814 may alter the prioritization of the search results inaddition to the removal of search results having scores below thespecific value. For example, utilization of the filter tool 814 maycause the search results to be presented via the website in a descendingorder based on score. In such a case, assuming a priority ordering ofleft-to-right and top-to-bottom, the website would present the searchresults in the following order: the first product (i.e., Product 1) 802,the eighth product (i.e., Product 8) 820, the seventh product (i.e.,Product 7) 818, the sixth product (i.e., Product 6) 812, the fourthproduct (i.e., Product 4) 808, and the third product (i.e., Product 3)806.

When the filter tool 814 is utilized, it causes a modified search queryto be generated. The modified search query is based on the search query(i.e., the original search query) and includes an indication of thefiltering. In one embodiment, the modified search query includesindications of products to be excluded from the search results.Returning to FIG. 7 and the example filtering of products having scoresbelow six, the modified search query would include an indication ofsecond product (i.e., Product 2) 704 and the fifth product (i.e.,Product 5) 710. The indication of the products can be any indicationsuitable for identifying the products. Continuing the example providedabove, if the original unmodified search query is “[search query],” themodified search query may be “[search query]+[exclude [UPC for thesecond product (i.e., Product 2) 704]]+[exclude [UPC for the fifthproduct (i.e., Product 5) 710]].”

As previously discussed with respect to FIG. 2, a system for presentingsearch diagnostic tools in concert with a website can include a keyvalue store. In embodiments in which such a system includes a key valuestore, when the user utilizes the filter tool to modify the searchquery, a mapping is created in the key value store that associates thesearch query with the modified search query.

While the discussion of FIGS. 7 and 8 describes a website presentationbefore and after the filter tool has been used, the discussion of FIG. 9describes example operations performed by a system for presenting searchdiagnostic tools in concert with a website when the filter tool is used.

FIG. 9 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments. The flow begins at block 902.

At block 902, code for a website is stored. For example, a server canstore the code for the website. The code for the website includesmultiple code sets. In one embodiment, the code for the website includestwo code sets: a first code set and a second code set. The first codeset includes executable code for rendering the website including asearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website. The user interface includes search diagnostic tools. The flowcontinues at block 904.

At block 904, filter values are stored. For example, a key value storeor database can store the filter values. The filter values areassociated with search queries. For example, if a user (e.g., anauthorized user) has turned on a filter tool for a specific searchquery, a filter value for that specific search query is stored in thedatabase. In some embodiments, the filter value can be set by the user.That is, the user can set whatever value he or she would like to use tofilter the search results. In some embodiments, the key value store canstore mappings between search queries and modified search queries. Forexample, the user may feel that the search results returned for a searchquery are different than what he or she would like. As will be discussedin detail below with respect to a filter tool, the user can alter theway the search results are presented on the website. When the user doesso, he or she indicates a modification to the search query. The resultis a modified search query. The modified search query is based on theoriginal (i.e., unmodified) search query. The key value store stores amapping between the original (i.e., unmodified) and modified searchquery. The flow continues at block 906.

At block 906, a search query is received. For example, a control circuitcan receive the search query from a user. The search query is input bythe user and seeks to find content on the website and/or on otherwebsites related to the search query. The search query is passed to thesearch engine to generate search results. The flow continues at block908.

At block 908, search results are received. For example, the searchresults are received by the control circuit from the search engine. Thesearch results are based on the search query. The flow continues atblock 910.

At block 910, signals are output based on the first code set. Forexample, the control circuit can output the signals based on the firstcode set. The signals based on the first code set cause rendering of thewebsite by the user device. The rendering of the website includes thesearch results (i.e., the results of the search query). The flowcontinues at block 912.

At block 912, signals are output based on the second code set. Forexample, the control circuit can output the signals based on the secondcode set. The signals based on the second code set cause rendering ofthe user interface including the search diagnostic tools such that theuser interface including the search diagnostic tools is presented withthe results for the search query. For example, the search diagnostictools can be presented next to, on top of, superimposed on, near,adjacent to, and/or in conjunction with the search results. The searchdiagnostic tools include a filter tool configured to control filteringof the results for the search query. The flow continues at block 914.

At block 914, an indication of a modification to the search query isreceived. For example, the control circuit can receive the indication ofthe modification to the search query. The indication of the modificationto the search query is based on utilization of the filter tool. Forexample, the modification to the search query can be the filtering ofsearch results below a value and/or indicate items to be excluded fromthe search results. The flow continues at block 916.

At block 916, a modified search query is generated. For example, thecontrol circuit can generate the modified search query. The modifiedsearch query is based on the search query and the indication of themodification of the search query. The search query can be modified inany manner suitable to obtain the desired filtering. For example, anindication of search results to exclude and/or the filter value can beincorporated into the search query to generate the modified searchquery. The flow continues at block 918.

At block 918, the modified search query is transmitted for storage. Forexample, the control circuit can transmit the modified search query tothe key value store for storage. Additionally, in some embodiments, thecontrol circuit can transmit the modified search query to a database(e.g., a SQL or NoSQL database) for storage. In such embodiments,additional information regarding the search query and the modifiedsearch query can be stored in the database. The flow continues at block920.

At block 920, submission of the search query is received from an enduser. For example, the control circuit can receive the submission of thesearch query from a user of the website, such as a general user or anauthorized user for testing and/or evaluation. The flow continues atblock 922.

At block 922, the modified search query is transmitted to the searchengine. For example, the control circuit can transmit the modifiedsearch query to the search engine. This step assumes that a modificationhas been performed to the search query submitted. For example, if amodified search query was previously generated for the specific searchquery that was submitted, instead of submitting the search query to thesearch engine, the modified search query is submitted. In someembodiments, the control circuit accesses the key value store toretrieve the modified search query associated with the search query. Insuch embodiments, the control circuit submits the modified search queryretrieved based on this access.

While the discussion of FIGS. 7-9 describes use of a filter tool of thesearch diagnostic tools, the discussion of FIGS. 10-11 describes use ofan explanation tool of the search diagnostic tools.

Explanation Tool (Explain):

FIG. 10 depicts a webpage being presented via a web browser 1000presenting an explain dialogue 1006, according to some embodiments. Insome embodiments, the search diagnostic tools include an explanationtool 1004. Although the explanation tool 1004 is associated with asecond product (i.e., Product 2) 1002, each search result depicted inFIG. 10 includes an explanation tool. When the explanation tool 1004 isselected (as indicated by the hashing), the explain dialogue 1006 ispresented. The explain dialogue 1006 includes explanations relating torendered search results. Accordingly, the explain dialogue 1006 caninclude any information relevant to why a specific search result wasreturned by the search engine. As one example, the explain dialogue 1006includes information relating to why a search result was returned (i.e.,why the product matched the search query) and how a search result isranked (i.e., how the product ranks based on the search query). Theinformation relating to why a search result was returned can be based onterms in the search query that match terms found in the search resulttitle, description, etc., thesaurus expansions of terms, categoricalinformation, etc. The information relating to how a search result isranked can be based on a number of terms matched, a number of categoriesmatched, the similarity between the search query and the product, etc.

Although FIG. 10 depicts only a single explanation dialogue 1006,embodiments are not so limited. For example, in some embodiments,multiple explanation dialogues can be presented simultaneously, for oneor multiple products. Additionally, as a further example, in someembodiments, the explanation tool 1004 can be global. That is, thesearch diagnostic tools can include one or more explanation tools 1004that cause explanation dialogues to be presented for all search results(e.g., all search results currently presented via the website). In suchembodiments, when the user scrolls through the search results, newexplanation dialogues can be presented as new search results arepresented.

Upon selection of the explanation tool 1004, a modified search query isgenerated. The modified search query includes a request for explanation.The request for explanation requests at least the information used togenerate the explanation dialogue 1006. Continuing the example describedabove, if the search query is “search query,” the modified search querymay be “search query+[explanation for [the second product (i.e., Product2) 1002]]” or “search query+[add explanations]+[add rankinginformation].” The modified search query is sent to the search engine.

In some embodiments, the explanation tool 1004 is available to allusers. For example, in such embodiments, no authorization information isrequired to access the explanation tool 1004. When the explanation tool1004 is available to all users, general users of the website (e.g.,shoppers) can gain a better understanding of why a search result wasreturned for a specific search query. In such embodiments, the code setthat includes the explanation tool 1004 may be separate from others ofthe search diagnostic tools. In some embodiments, the explanation tool1004 is available to all users because it does not modify search queriesfor subsequent users (i.e., it can be thought of as “read only”).

While the discussion of FIG. 10 provides background informationregarding an explanation tool of the search diagnostic tools, thediscussion of FIG. 11 describes example operations performed by a systemfor presenting search diagnostic tools in concert with a website whenthe explanation tool is used.

FIG. 11 is a flow chart including example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments. The flow begins at block 1102.

At block 1102, code for a website is stored. For example, a server canstore the code for the website. The code for the website includesmultiple code sets. In one embodiment, the code for the website includestwo code sets: a first code set and a second code set. The first codeset includes executable code for rendering the website including asearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website. The user interface includes search diagnostic tools. The flowcontinues at block 11004.

At block 1104, a search query is received. For example, a controlcircuit can receive the search query from a user. The search query isinput by the user and seeks to find content on the website and/or onother websites related to the search query. The search query is passedto the search engine to generate search results. The flow continues atblock 1106.

At block 1106, search results are received. For example, the searchresults are received by the control circuit from the search engine. Thesearch results are based on the search query. The flow continues atblock 1108.

At block 1108, signals are output based on the first code set. Forexample, the control circuit can output the signals based on the firstcode set. The signals based on the first code set cause rendering of thewebsite by the user device. The rendering of the website includes thesearch results (i.e., the results of the search query). The flowcontinues at block 1110.

At block 1110, signals are output based on the second code set. Forexample, the control circuit can output the signals based on the secondcode set. The signals based on the second code set cause rendering ofthe user interface including the search diagnostic tools such that theuser interface including the search diagnostic tools is presented withthe results for the search query. For example, the search diagnostictools can be presented next to, on top of, superimposed on, near,adjacent to, and/or in conjunction with the search results. The searchdiagnostic tools include an explanation tool configured to receivediagnostic requests for explanations relating to rendered searchresults. As previously discussed, the explanation tool may be accessibleto users that do not have authorization information. In suchembodiments, at least some of the search diagnostic tools may beincluded in a third code set. The flow continues at block 1112.

At block 1112, an indication of a modification to the search query isreceived. For example, the control circuit can receive the indication ofthe modification to the search query. The indication of the modificationto the search query is based on utilization of the explanation tool. Forexample, the modification to the search query can be a request forexplanations relating to rendered search results. The flow continues atblock 1114.

At block 1114, a modified search query is generated. For example, thecontrol circuit can generate the modified search query. The modifiedsearch query is based on the search query and the indication of themodification of the search query. The search query can be modified inany manner suitable to obtain the desired explanation. For example,request for an explanation as to why one or more search results wereincluded can be incorporated into the search query to generate themodified search query. The flow continues at block 1116.

At block 1116, the modified search query is transmitted to the searchengine. For example, the control circuit can transmit the modifiedsearch query to the search engine. That is, once the modified searchquery is generated, the modified search query is transmitted to thesearch engine. The search engine returns the information relating to whyone or more search results were included. The information relating towhy one or more search results were included can include an NDCG score,a list of matching terms, a ranking, a summary, a list of matchingproducts, a list of matching records, a summary for textual matching ofterms to records across various fields a record's data, a summary of aranking of a record among others who match similarly, spell corrections,synonym expansions, and stemming.

While the discussion of FIGS. 10-11 describes use of an explanation toolof the search diagnostic tools, the discussion of FIGS. 12-13 describesuse of a query tool of the search diagnostic tools.

Query Tool (Query Runner):

An automated query inquiry helps to automate the process of a useranalyzing search queries. For example, a user would typically analyze asearch query by typing the search query into a search bar and executingthe search. The user would then review the search results returned basedon the search query. If a user wanted to analyze a number of searchqueries, the user would need to repeat this process for each searchquery analyzed. In practice, a data analyst would compile a list ofsearch queries (i.e., a “set of queries”) to be analyzed. The user wouldthen take the list and, one-by-one, enter the queries into the website,receive the search results, and review the search results. Depending onthe length of the list, this process can be quite time consuming.Additionally, manually retrieving search queries, entering searchqueries, and searching based on search queries accounts for asubstantial portion of the time required to execute the task.

The query tool helps automate much of this process. In some embodiments,the query tool is capable of acquiring a set of queries andautomatically running searches associated with the queries in the set ofqueries. That is, the query tool iterates through the queries in the setof queries automatically for the user. During this process, the user canview the results associated with the search queries and take control ofthe process if he or she sees fit. For example, when the automated queryinquiry is running, if the user sees search results that he or she feelsare not appropriate for the search query, he or she can pause theautomated query inquiry. Once the automated query inquiry is paused, theuser can use any of the search diagnostic tools that he or she wishes toalter the search results based on the search query. The discussion ofFIGS. 12-13 provides additional detail regarding such a query tool.

FIG. 12 depicts a webpage being presented via a web browser 1200including a query tool 1204, according to some embodiments. The querytool 1204 includes controls for running an automated query inquiry. Insome embodiments, the controls for running the automated query resemblethe type of controls that would be found on a media player. For example,the controls for running an automated query inquiry depicted in FIG. 12include a previous button 1206, a play button 1210, a pause button 1212,a next button 1214, and a shuffle button 1216. The previous button 1206backs up to a previous query, the play button 1210 begins the automatedquery inquiry and/or resumes the automated query inquiry, the pausebutton 1212 pauses the automated query inquiry, the next button 1214advances to a next query in the set of queries, and the shuffle button1216 causes random selection of the queries from the set of queries.

In some embodiments, the controls for running the automated queryinquiry also allow the user to select a specific query from the set ofqueries (i.e., a select button). For example, as depicted in FIG. 12,the user has selected the query “Query 2” 1218 from list of queries1220. The list of queries 1220 presents at least some of the queriesincluded in the set of queries. In some embodiments, the list of queries1220 also includes information about queries in the set of queries. Forexample, as depicted in FIG. 12, the list of queries 1220 includes ascore associated with each query (e.g., an average score of all searchresults for the query) and a number of searches performed using a searchquery (e.g., over a time period).

The list of queries 1220 can be populated in any suitable manner. Insome embodiments, the query tool 1204 automatically populates the listof queries 1220. For example, the query tool 1204 can determine queriesto include in the list of queries 1220 (i.e., which queries to includein a set of queries), with and/or without user input. In embodiments inwhich the user participates in the population of the set of queries, theuser can specify which types of queries to include in the set ofqueries. In either case (i.e., with or without user input), the querytool 1204 can determine the set of queries based on any suitablecriteria. For example, the query tool 1204 can determine the set ofqueries based on frequency of a query, volume of a query, average scorefor a query, revenue generated based on a query, margins for sales basedon a query, click-through-rate for the query, add-to-cart rate for thequery, page view rate, user rating, or any other suitable rankingcriteria. In some embodiments, the query tool 1204 requests the set ofqueries from an analytics tool. For example, the query tool 1204 canrequest that the analytics tool provide a set of queries including the100 most frequently searched queries.

While the discussion of FIG. 12 provides background informationregarding a query tool, the discussion of FIG. 13 describes exampleoperations for the query tool.

FIG. 13 is a flow chart depicting example operations for presentingsearch diagnostic tools in concert with a website, according to someembodiments. The flow begins at block 1302.

At block 1302, code for a website is stored. For example, a server canstore the code for the website. The code for the website includesmultiple code sets. In one embodiment, the code for the website includestwo code sets: a first code set and a second code set. The first codeset includes executable code for rendering the website including asearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website. The user interface includes search diagnostic tools. The flowcontinues at block 1304.

At block 1304, signals are output based on the first code set. Forexample, the control circuit can output the signals based on the firstcode set. The signals based on the first code set cause rendering of thewebsite by the user device. The rendering of the website includes thesearch results (i.e., the results of the search query). The flowcontinues at block 1306.

At block 1306, signals are output based on the second code set. Forexample, the control circuit can output the signals based on the secondcode set. The signals based on the second code set cause rendering ofthe user interface including the search diagnostic tools such that theuser interface including the search diagnostic tools is presented withthe results for the search query. For example, the search diagnostictools can be presented next to, on top of, superimposed on, near,adjacent to, and/or in conjunction with the search results. The searchdiagnostic tools include a query tool including controls for running anautomated query inquiry. The flow continues at block 1308.

At block 1308, user input is received to begin an automated queryinquiry. For example, the user input can be received by the controlcircuit via the user interface. That is, the user can utilize the querytool to begin the automated query inquiry. Additionally, in embodimentsin which the query tool includes additional controls, the user canperform other actions during the automated query inquiry, such aspausing the automated query inquiry. The flow continues at block 1310.

At block 1310, a set of queries is determined. For example, the controlcircuit can determine the set of queries. In some embodiments, thecontrol circuit determines the set of queries based on user inputdefining types of queries to be included in the query set. For example,the user may indicate that he or she wants to include the top 100queries by volume in the query set. Additionally, in some embodiments,the query set can be generated by an analytics tool. In suchembodiments, the control circuit requests the set of queries from theanalytics tool.

After the set of queries is determined, the flow continues by iteratingthrough the queries in the set of queries. These operations are depictedin block 1314-1318. That is, for each of the queries in the query set(assuming the user has not taken over control and ended the automatedquery inquiry), the operations depicted in blocks 1314-1318 areperformed automatically.

At block 1314, a search query is transmitted. For example, the controlcircuit can transmit the search query to a search engine. The searchquery is one of the queries from the query set. The flow continues atblock 1316.

At block 1316, search results are received. For example, the controlcircuit can receive the search results from the search engine. Thesearch results are based on the search query from the set of queries.The flow continues at block 1318.

At block 1318, signals are output to cause rendering of the websiteincluding the search results and the user interface. For example, thecontrol circuit can output the signals to cause rendering of the websiteincluding the search results and the user interface. As the searchqueries in the set of queries are iterated through, the user ispresented, via the website, with the search results for the queries. Theuser can take control of the automated query inquiry if he or shechooses. The user can take control by simply pausing the automated queryinquiry, moving to the next, or previous, query in the query set,changing the order in which the automated query inquiry iterates throughthe set of queries, etc. Additionally, in some embodiments, the user canutilize others of the search diagnostic tools to modify the searchqueries. For example, if during the automated query inquiry the usersees search results that he or she does not think belong or are notrelevant, he or she can take control of the automated query inquiry andalter the search results.

CONCLUSION

Described herein are systems, methods, and apparatuses for presentingsearch diagnostic tools in concert with a website. While a number ofspecific search diagnostic tools are presented separately, it should benoted that the search diagnostic tools presented in concert with a website can include any one of the aforementioned search diagnostic tools,any combination of multiple ones of the search diagnostic tools, and/orany other search diagnostic tools not described herein. Additionally,while most examples described herein relate to a website for sellinggoods and/or services, embodiments are not so limited. The systems,apparatuses, and methods described herein can be incorporated with anywebsite that provides a search function.

Although the examples of the systems, methods, and apparatuses usedherein refer to the search functionality of a website, in someembodiments, these systems, methods, and apparatuses can be used withsearch functions that are not related to or incorporated with websites.For example, the search diagnostic tools described herein can be used tosearch a database or other repository. In such embodiments, the userinterface provides the search diagnostic tools for use with the databaseor other repository. Put simply, the user interface and searchdiagnostic tools described herein can be incorporated with any systemthat allows users to search for data.

In some embodiments, a system for presenting search diagnostic tools inconcert with a website comprises a server, wherein the server storescode for a website including a search function for users of the website,wherein the code for the website includes a first code set and a secondcode set, wherein the first code set includes executable code forrendering the website including the search function and the second codeset includes executable code for rendering a user interface integratedwith the rendering of the website, and wherein the user interfaceincludes search diagnostic tools for the website, and a control circuit,wherein the control circuit is configured to receive, from a user, asearch query, receive, from a search engine, results for the searchquery, output based on execution of the first code set, signals to causerendering of the website, wherein the website includes the results forthe search query, and output, based on execution of the second code set,signals to cause rendering of the user interface including the searchdiagnostic tools such that the user interface including the searchdiagnostic tools is presenting with the search results for the searchquery.

In some embodiments, an apparatus and a corresponding method performedby the apparatus comprises storing, at a server, code for a websiteincluding a search function for users of the website, wherein the codefor the website includes a first code set and a second code set, whereinthe first code set includes executable code for rendering the websiteincluding the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchdiagnostic tools for the website, receiving, at a control circuit from auser, a search query, receiving, at the control circuit from a searchengine, results for the search query, outputting, by the control circuitbased on execution of the first code set, signals to cause rendering ofthe website, wherein the website includes the results for the searchquery, and outputting, by the control circuit based on execution of thesecond code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search diagnostic tools is presenting with the searchresults for the search query.

In some embodiments, a system for modifying search queries comprises aserver, wherein the server stores code for a website including a searchfunction for users of the website, wherein the code for the web siteincludes a first code set and a second code set, and wherein the firstcode set includes executable code for rendering the website includingthe search function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website,and wherein the user interface includes search diagnostic tools for theweb site, a control circuit, wherein the control circuit is configuredto receive, from a user, a search query, receive, from a search engine,results for the search query, output based on execution of the firstcode set, signals to cause rendering of the website, wherein the websiteincludes the results for the search query, output, based on execution ofthe second code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search diagnostic tools is presented with the searchresults for the search query, wherein the search diagnostic toolsinclude a search explanation tool configured to receive diagnosticrequests for explanations relating to rendered search results, receive,via the search explanation tool, an indication of a modification to thesearch query, wherein the modification to the search query incudes arequest for explanation, generate, based on the indication of themodification for the search query, a modified search query, and wherein,upon submission of the search query by the user, the search enginereceives the modified search query.

In some embodiments of the system, the request for explanation requestsan explanation from the search engine as to why one or more results forthe search query were included.

In some embodiments of the system, in response to receipt of the requestfor explanation, the search engine provides one or more of a normalizeddiscounted cumulative gain (NDCG) score, a list of matching terms, aranking, summary, a list of matching products, a list of matchingrecords, a summary for textual matching of terms to records acrossvarious fields a record's data, a summary of a ranking of a record amongothers who match similarly, spell corrections, synonym expansions, andstemming

In some embodiments of the system, the user interface includes anexplanation selection for each of the results for the search query, andwherein the explanation selection for each of the results for the searchquery is presented in conjunction with one of the results for the searchquery.

In some embodiments of the system, the explanation selection for each ofthe results of the search query is viewable by the user and an end user.

In some embodiments of the system, the control circuit is furtherconfigured to receive, from the user, authorization information andconfirm, based on the authorization information, that the user isauthorized to access the second code set.

In some embodiments of the system, the authorization informationincludes a username and a password.

In some embodiments of the system, the control circuit is furtherconfigured to receive, by the control circuit in response to themodified search query, data explaining why a search result was includedin the results for the search and output signals to cause rendering of adialogue, wherein the dialogue includes the data explaining why a searchresult was included in the results for the search.

In some embodiments of the system, the system further comprises thesearch engine, wherein the search engine is configured to, upon receiptof the modified search query, return data explaining why a search resultwas included in the results for the search.

In some embodiments, an apparatus and a corresponding method performedby the apparatus comprises storing, at a server, code for a websiteincluding a search function for users of the website, wherein the codefor the website includes a first code set and a second code set, whereinthe first code set includes executable code for rendering the websiteincluding the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchdiagnostic tools for the website, receiving, at a control circuit from auser, a search query, receiving, at the control circuit from a searchengine, results for the search query, outputting, by the control circuitbased on execution of the first code set, signals to cause rendering ofthe website, wherein the website includes the results for the searchquery, outputting, by the control circuit based on execution of thesecond code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search modifications tools is presented with the searchresults for the search query, wherein the search diagnostic toolsinclude a search explanation tool configured to receive diagnosticrequests for explanations relating to rendered search results,receiving, via the search explanation tool, an indication of amodification to the search query, wherein the modification to the searchquery incudes a request for explanation, generating, based on theindication of the modification for the search query, a modified searchquery, receiving, at the control circuit, submission of the searchquery, and transmitting, to the search engine in response to thereceiving the search query, the modified search query.

In some embodiments of the method, the request for explanation requestsan explanation from the search engine as to why one or more results fromthe search query were included.

In some embodiments of the method, in response to receipt of the requestfor explanation, the search engine provides one or more of a normalizeddiscounted cumulative gain (NDCG) score, a list of matching terms, aranking, summary, a list of matching products, a list of matchingrecords, a summary for textual matching of terms to records acrossvarious fields a record's data, a summary of a ranking of a record amongothers who match similarly, spell corrections, synonym expansions, andstemming.

In some embodiments of the method, the user interface includes anexplanation selection for each of the results for the search query, andwherein the explanation selection for each of the results for the searchquery is presented in conjunction with one of the results for the searchquery.

In some embodiments of the method, the explanation selection for each ofthe results of the search query is viewable by the user and an end user.

In some embodiments, the method further comprises receiving, from theuser, authorization information and confirming, based on theauthorization information, that the user is authorized to access thesecond code set.

In some embodiments of the method, the authorization informationincludes a username and a password.

In some embodiments, the method further comprises receiving, by thecontrol circuit in response to the modified search query, dataexplaining why a search result was included in the results for thesearch and outputting signals to cause rendering of a dialogue, whereinthe dialogue includes the data explaining why a search result wasincluded in the results for the search.

In some embodiments, the method further comprises returning, by thesearch engine in response to receipt of the modified search query, dataexplaining why a search result was included in the results for thesearch.

In some embodiments, a system for modifying search queries comprises aserver, wherein the server stores code for a website including a searchfunction for users of the website, wherein the code for the web siteincludes a first code set and a second code set, wherein the first codeset includes executable code for rendering the website including thesearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website,and wherein the user interface includes search diagnostic tools for thewebsite, a key value store, wherein the key value store includesmodified search queries corresponding to filter values, and a controlcircuit, wherein the control circuit is configured to receive, from auser, a search query, receive, from a search engine, results for thesearch query, output based on execution of the first code set, signalsto cause rendering of the website, wherein the website includes theresults for the search query, output, based on execution of the secondcode set, signals to cause rendering of the user interface including thesearch diagnostic tools such that the user interface including thesearch diagnostic tools is presented with the search results for thesearch query, wherein the search diagnostic tools include a filter toolconfigured to control filtering of the results for the search query,receive, via the filter tool, an indication of a modification to thesearch query, wherein the modification to the search query includes afiltering of the results for the search query, generate, based on theindication of the modification for the search query, a modified searchquery, transmit, for storage in the key value store, an indication ofthe modification for the search query, wherein, upon submission of thesearch query by an end user, the search engine receives the modifiedsearch query.

In some embodiments of the system, the filter values are normalizeddiscounted cumulative gain (NDCG) scores.

In some embodiments of the system, the filtering of the results for thesearch query is based on one of the NDCG scores included in the keyvalue store.

In some embodiments of the system, the user interface includes afiltering toggle, wherein selection of the filtering toggle turns thefiltering of the results for the search query on and off.

In some embodiments of the system, search results based on the modifiedsearch query exclude ones of the results for the search query havingvalues below a threshold.

In some embodiments of the system, the control circuit is furtherconfigured to receive, via the user interface including the searchdiagnostic tools, an indication of a filter value for the search query.

In some embodiments of the system, the modified search query includesthe indication of the filter value for the search query.

In some embodiments of the system, the control circuit is furtherconfigured to receive, from the user, authorization information andconfirm, based on the authorization information, that the user isauthorized to access the second code set.

In some embodiments of the system, the authorization informationincludes a username and a password.

In some embodiments, the system further comprises a client device,wherein the end user submits the search query via the client device, andwherein the client device is configured to present the website includingmodified search results based on the modified search query.

In some embodiments, an apparatus and a corresponding method performedby the apparatus comprises storing, at a server, code for a websiteincluding a search function for users of the website, wherein the codefor the website includes a first code set and a second code set, whereinthe first code set includes executable code for rendering the websiteincluding the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchmodification tools for the website, storing, at a key value store,filter values for search queries, receiving, at a control circuit from auser, a search query, receiving, at the control circuit from a searchengine, results for the search query, outputting, by the control circuitbased on execution of the first code set, signals to cause rendering ofthe website, wherein the website includes the results for the searchquery, outputting, by the control circuit based on execution of thesecond code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search modifications tools is presented with the searchresults for the search query, wherein the search diagnostic toolsinclude a filter tool configured to control filtering of the results forthe search query, receiving, via the filter tool, an indication of amodification to the search query, wherein the modification to the searchinquiry includes a filtering of the results for the search query,generating, based on the indication of the modification for the searchquery, a modified search query, transmitting, for storage in the keyvalue store, the modified search query, receiving, at the controlcircuit, submission of the search query from an end user, andtransmitting, to the search engine in response to the receiving thesearch query, the modified search query.

In some embodiments of the method, the filter values for search queriesare normalized discounted gain (NDCG) scores.

In some embodiments of the method, the filtering of the results for thesearch query is based on one of the NDCG scores in a database.

In some embodiments of the method, the user interface includes afiltering toggle, and wherein selection of the filtering toggle turnsthe filtering of the results for the search query on and off.

In some embodiments of the method, search results based on the modifiedsearch query exclude ones of the results for the search query havingvalues below a threshold.

In some embodiments, the method further comprises receiving, via theuser interface including the search diagnostic tools, an indication of afilter value for the search query.

In some embodiments of the method, the modified search query includesthe indication of the filter value for the search query.

In some embodiments, the method further comprises receiving, from theuser, authorization information and confirming, based on theauthorization information, that the user is authorized to access thesecond code set.

In some embodiments of the method, the authorization informationincludes a username and a password.

In some embodiments, the method further comprises receiving, at a clientdevice, submission of the search query and presenting, by the clientdevice, the website including modified search results based on themodified search query.

In some embodiments, a system for presenting search diagnostic tools inconcert with a website comprises a server, wherein the server storescode for a website including a search function for users of the website,wherein the code for the website includes a first code set and a secondcode set, wherein the first code set includes executable code forrendering the website including the search function and the second codeset includes executable code for rendering a user interface integratedwith the rendering of the website, and wherein the user interfaceincludes search diagnostic tools for the website, a key value store,wherein the key value store includes a mapping between search queriesand modified search queries, and a control circuit, wherein the controlcircuit is configured to receive, from a user, a search query, receive,from a search engine, results for the search query, output based onexecution of the first code set, signals to cause rendering of thewebsite, wherein the website includes the results for the search query,output, based on execution of the second code set, signals to causerendering of the user interface including the search diagnostic toolssuch that the user interface including the search diagnostic tools ispresented with the search results for the search query, wherein thesearch diagnostic tools include a prioritization tool configured tocontrol prioritization of the results for the search query, receive, viathe prioritization tool, an indication of a modification to the searchquery, wherein the modification to the search query includes one ofboosting a search result and burying a search result, generate, based onthe indication of the modification for the search query, a modifiedsearch query, transmit, for storage in the key value store, the modifiedsearch query, wherein, upon submission of the search query by an enduser, the search engine receives the modified search query.

In some embodiments of the system, the modification for the search queryis a boost of a first search result, and wherein the modified searchquery positively includes a product identifier for the first searchresult.

In some embodiments of the system, the modification for the search queryis a bury of a first search result, and wherein the modified searchquery negatively includes a product identifier for the first searchresult.

In some embodiments of the system, the search diagnostic tools include afirst selection and a second selection, wherein the first selection andthe second selection are associated with a first search result, andwherein selection of one of the first selection and the second selectioncauses the first search result to be one of boosted and buried.

In some embodiments of the system, the control circuit is furtherconfigured to receive, from the user, authorization information andconfirm, based on the authorization information, that the user isauthorized to access the second code set.

In some embodiments of the system, the authorization informationincludes a username and a password.

In some embodiments of the system, the control circuit is furtherconfigured to determine, upon submission of the search query by the enduser based on access to the key value store, the modified search query,wherein the control circuit transmits the modified search query to thesearch engine.

In some embodiments the system further comprises a user device, whereinthe end user submits the search query via the user device, and whereinthe user device is configured to present the website including modifiedsearch results based on the modified search query.

In some embodiments, the system further comprises a database, whereinthe database stores the search queries, the modified search queries, andinformation related to the search queries and the modified searchqueries.

In some embodiments of the system, the control circuit is furtherconfigured to save, in the database, an association between the searchquery and the modified search query.

In some embodiments, an apparatus and a corresponding method performedby the apparatus comprises storing, at a server, code for a websiteincluding a search function for users of the website, wherein the codefor the website includes a first code set and a second code set, whereinthe first code set includes executable code for rendering the websiteincluding the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchmodification tools for the website, storing, at a key value store, amapping between search queries and modified search queries, receiving,at a control circuit from a user, a search query, receiving, at thecontrol circuit from a search engine, results for the search query,outputting, by the control circuit based on execution of the first codeset, signals to cause rendering of the website, wherein the websiteincludes the results for the search query, outputting, by the controlcircuit based on execution of the second code set, signals to causerendering of the user interface including the search diagnostic toolssuch that the user interface including the search diagnostic tools ispresented with the search results for the search query, wherein thesearch diagnostic tools include a prioritization tool configured tocontrol prioritization of the results for the search query, receiving,via the prioritization tool, an indication of a modification to thesearch query, wherein the modification to the search inquiry includesone of boosting a search result and burying a search result, generating,based on the indication of the modification for the search query, amodified search query, transmitting, for storage in the key value store,the modified search query, receiving, at the control circuit, submissionof the search query from an end user, transmitting, to the search enginein response to the receiving the search query, the modified searchquery.

In some embodiments of the method, the modification for the search queryis a boost of a first search result, and wherein the modified searchquery positively includes a product identifier for the first searchresult.

In some embodiments of the method, the modification for the search queryis a bury of a first search result, and wherein the modified searchquery negatively includes a product identifier for the first searchresult.

In some embodiments of the method, the search diagnostic tools include afirst selection and a second selection, wherein the first selection andthe second selection are associated with a first search result, andwherein selection of one of the first selection and the second selectioncauses the first search result to be one of boosted and buried.

In some embodiments, the method further comprises receiving, from theuser, authorization information and confirming, based on theauthorization information, that the user is authorized to access thesecond code set.

In some embodiments of the method, the authorization informationincludes a username and a password.

In some embodiments, the method further comprises determining, uponsubmission of the search query by the end user based on access to thekey value store, the modified search query, wherein the control circuittransmits the modified search query to the search engine.

In some embodiments, the method further comprises receiving, at a userdevice, submission of the search query and presenting, by the userdevice, the website including modified search results based on themodified search query.

In some embodiments, the method further comprises storing, in adatabase, the search queries, the modified search queries, andinformation related to the search queries and the modified searchqueries.

In some embodiments, the method further comprises saving, in thedatabase, an association between the search query and the modifiedsearch query.

In some embodiments, a system for presenting search diagnostic tools inconcert with a website comprises a server, wherein the server storescode for a website including a search function for users of the website,wherein the code for the website includes a first code set and a secondcode set, wherein the first code set includes executable code forrendering the website including the search function and the second codeset includes executable code for rendering a user interface integratedwith the rendering of the website, and wherein the user interfaceincludes search diagnostic tools for the website, a control circuit,wherein the control circuit is configured to output based on executionof the first code set, signals to cause rendering of the website,output, based on execution of the second code set, signals to causerendering of the user interface including the search diagnostic toolssuch that the user interface including the search diagnostic tools ispresented with the website, wherein the search diagnostic tools includea query tool, and wherein the query tool includes controls for runningan automated query inquiry, receive, via the user interface from a user,user input to begin the automated query inquiry, determine, in responseto the selection of the query tool, a set of queries, for each query inthe set of queries until further user input is received transmit, to asearch engine, the query, receive, from the search engine, queryresults, and output, based on the query results, signals to causerendering of the website including the query results and the userinterface including the search diagnostic tools.

In some embodiments of the system, the controls for running theautomated query inquiry include a play button, a pause button, a nextbutton, a previous button, a shuffle button, and a select button.

In some embodiments of the system, the play button begins the automatedquery inquiry, the pause button pauses the automated query inquiry, thenext button advances to a next query in the set of queries, the previousbutton backs up to a previous query, the shuffle button causes randomselection of queries from the set of queries, and the select buttonallows selection of a specific query from the set of queries.

In some embodiments of the system, the search diagnostic tools includeone or more of a boost tool, a bury tool, a filtering tool, and arequest for explanation tool.

In some embodiments of the system, the user is able to perform at leastone of the search diagnostic tools when the automated query inquiry ispaused.

In some embodiments of the system, the control circuit is furtherconfigured to receive, from the user, authorization information andconfirm, based on the authorization information, that the user isauthorized to access the second code set.

In some embodiments of the system, the authorization informationincludes a username and a password.

In some embodiments of the system, the control circuit determines theset of queries based on one or more of frequency, volume, average score,revenue, margins, click-through-rate, and add-to-cart rate, page viewrate, and user rating.

In some embodiments, the system further comprises an analytics tool,wherein the analytics tool is configured to receive, from the controlcircuit, a request for the set of queries, generate, based on therequest for the set of queries, the set of queries, and transmit, to thecontrol circuit, the set of queries.

In some embodiments, an apparatus and a corresponding method performedby the apparatus comprises storing, at a server, code for a websiteincluding a search function for users of the website, wherein the codefor the website includes a first code set and a second code set, whereinthe first code set includes executable code for rendering the websiteincluding the search function and the second code set includesexecutable code for rendering a user interface integrated with therendering of the website, and wherein the user interface includes searchdiagnostic tools for the website, outputting, by the control circuitbased on execution of the first code set, signals to cause rendering ofthe website, outputting, by the control circuit based on execution ofthe second code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search diagnostic tools are presented with the website,wherein the search diagnostic tools include a query tool, and whereinthe query tool includes controls for running an automated query inquiry,receiving, via the user interface from a user, user input to begin theautomated query inquiry, determining, in response to the selection ofthe query tool, a set of queries, for each query in the set of queriesuntil further user input is received transmitting, to a search engine,the query, receiving, from the search engine, query results, andoutputting, based on the query results, signals to cause rendering ofthe website including the query results and the user interface includingthe search diagnostic tools.

In some embodiments of the method, the controls for running theautomated query inquiry include a play button, a pause button, a nextbutton, a previous button, a shuffle button, and a select button.

In some embodiments of the method, the play button begins the automatedquery inquiry, the pause button pauses the automated query inquiry, thenext button advances to a next query in the set of queries, the previousbutton backs up to a previous query, the shuffle button causes randomselection of queries from the set of queries, and the select buttonallows selection of a specific query from the set of queries.

In some embodiments of the method, the search diagnostic tools includeone or more of a boost tool, a bury tool, a filtering tool, and arequest for explanation tool.

In some embodiments of the method, the user is able to perform at leastone of the search diagnostic tools when the automated query inquiry ispaused.

In some embodiments, the method further comprises receiving, from theuser, authorization information and confirming, based on theauthorization information, that the user is authorized to access thesecond code set.

In some embodiments of the method, wherein the authorization informationincludes a username and a password.

In some embodiments of the method, the control circuit determines theset of queries based on based on one or more of frequency, volume,average score, revenue, margins, click-through-rate, add-to-cart rate,page view rate, and user rating.

In some embodiments, the method further comprises receiving, by ananalytics tool, a request for a set of queries, generating, based on therequest for the set of queries, the set of queries, and transmitting, tothe control circuit, the set of queries.

Those skilled in the art will recognize that a wide variety of othermodifications, alterations, and combinations can also be made withrespect to the above described embodiments without departing from thescope of the invention, and that such modifications, alterations, andcombinations are to be viewed as being within the ambit of the inventiveconcept.

What is claimed is:
 1. A system for presenting search diagnostic toolsin concert with a website, the system comprising: a server, wherein theserver stores code for a website including a search function for usersof the website, wherein the code for the website includes a first codeset and a second code set, wherein the first code set includesexecutable code for rendering the website including the search functionand the second code set includes executable code for rendering a userinterface integrated with the rendering of the website, and wherein theuser interface includes search diagnostic tools for the web site; and akey value store, wherein the key value store includes a mapping betweensearch queries and modified search queries; and a control circuit,wherein the control circuit is configured to: receive, from a user, asearch query; receive, from a search engine, results for the searchquery; output based on execution of the first code set, signals to causerendering of the website, wherein the website includes the results forthe search query; output, based on execution of the second code set,signals to cause rendering of the user interface including the searchdiagnostic tools such that the user interface including the searchdiagnostic tools is presented with the search results for the searchquery, wherein the search diagnostic tools include a prioritization toolconfigured to control prioritization of the results for the searchquery; receive, via the prioritization tool, an indication of amodification to the search query, wherein the modification to the searchquery includes one of boosting a search result and burying a searchresult; generate, based on the indication of the modification for thesearch query, a modified search query; and transmit, for storage in thekey value store, the modified search query; wherein, upon submission ofthe search query by an end user, the search engine receives the modifiedsearch query.
 2. The system of claim 1, wherein the modification for thesearch query is a boost of a first search result, and wherein themodified search query positively includes a product identifier for thefirst search result.
 3. The system of claim 1, wherein the modificationfor the search query is a bury of a first search result, and wherein themodified search query negatively includes a product identifier for thefirst search result.
 4. The system of claim 1, wherein the searchdiagnostic tools include a first selection and a second selection,wherein the first selection and the second selection are associated witha first search result, and wherein selection of one of the firstselection and the second selection causes the first search result to beone of boosted and buried.
 5. The system of claim 1, wherein the controlcircuit is further configured to: receive, from the user, authorizationinformation; and confirm, based on the authorization information, thatthe user is authorized to access the second code set.
 6. The system ofclaim 5, wherein the authorization information includes a username and apassword.
 7. The system of claim 1, wherein the control circuit isfurther configured to: determine, upon submission of the search query bythe end user based on access to the key value store, the modified searchquery, wherein the control circuit transmits the modified search queryto the search engine.
 8. The system of claim 1, further comprising: auser device, wherein the end user submits the search query via the userdevice, and wherein the user device is configured to: present thewebsite including modified search results based on the modified searchquery.
 9. The system of claim 1, further comprising: a database, whereinthe database stores the search queries, the modified search queries, andinformation related to the search queries and the modified searchqueries.
 10. The system of claim 1, wherein the control circuit isfurther configured to: save, in the database, an association between thesearch query and the modified search query.
 11. A method for presentingsearch diagnostic tools in concert with a web site, the methodcomprising: storing, at a server, code for a web site including a searchfunction for users of the web site, wherein the code for the web siteincludes a first code set and a second code set, wherein the first codeset includes executable code for rendering the website including thesearch function and the second code set includes executable code forrendering a user interface integrated with the rendering of the website, and wherein the user interface includes search modification toolsfor the website; storing, at a key value store, a mapping between searchqueries and modified search queries; receiving, at a control circuitfrom a user, a search query; receiving, at the control circuit from asearch engine, results for the search query; outputting, by the controlcircuit based on execution of the first code set, signals to causerendering of the website, wherein the website includes the results forthe search query; outputting, by the control circuit based on executionof the second code set, signals to cause rendering of the user interfaceincluding the search diagnostic tools such that the user interfaceincluding the search diagnostic tools is presented with the searchresults for the search query, wherein the search diagnostic toolsinclude a prioritization tool configured to control prioritization ofthe results for the search query; receiving, via the prioritizationtool, an indication of a modification to the search query, wherein themodification to the search inquiry includes one of boosting a searchresult and burying a search result; generating, based on the indicationof the modification for the search query, a modified search query;transmitting, for storage in the key value store, the modified searchquery; receiving, at the control circuit, submission of the search queryfrom an end user; and transmitting, to the search engine in response tothe receiving the search query, the modified search query.
 12. Themethod of claim 11, wherein the modification for the search query is aboost of a first search result, and wherein the modified search querypositively includes a product identifier for the first search result.13. The method of claim 11, wherein the modification for the searchquery is a bury of a first search result, and wherein the modifiedsearch query negatively includes a product identifier for the firstsearch result.
 14. The method of claim 11, wherein the search diagnostictools include a first selection and a second selection, wherein thefirst selection and the second selection are associated with a firstsearch result, and wherein selection of one of the first selection andthe second selection causes the first search result to be one of boostedand buried.
 15. The method of claim 11, further comprising: receiving,from the user, authorization information; and confirming, based on theauthorization information, that the user is authorized to access thesecond code set.
 16. The method of claim 15, wherein the authorizationinformation includes a username and a password.
 17. The method of claim11, further comprising: determining, upon submission of the search queryby the end user based on access to the key value store, the modifiedsearch query, wherein the control circuit transmits the modified searchquery to the search engine.
 18. The method of claim 11, furthercomprising: receiving, at a user device, submission of the search query;and presenting, by the user device, the website including modifiedsearch results based on the modified search query.
 19. The method ofclaim 11, further comprising: storing, in a database, the searchqueries, the modified search queries, and information related to thesearch queries and the modified search queries.
 20. The method of claim19, further comprising: saving, in the database, an association betweenthe search query and the modified search query.